Testing Microcontroller Software Simulators

Software simulators that emulate equivalent behavior of physical microcontrollers play an important role in the process of software development for embedded systems from an early development stage (e.g. when no target hardware is available) to the final verification process (e.g. used in combination with formal methods). Thus, much reliance is put on the correctness of these simulators. This paper presents a practicable approach to test auto-generated and custom microcontroller simulators (both closed and open-source) against a physical device. We show how to set up a test oracle that allows to run the simulators in parallel, validate individual runs based on a comparison of their accumulated state-space, and – in case an error is found – finger-point to the root cause of the error, thus giving valuable support for fixing the discrepancies. A case study shows that the presented testing framework was able to reveal non-trivial bugs in several implementations.